{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2509fcb7-f84e-4c01-b23c-6e0942c2df10",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Chat Engine - ReAct Mode"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84aca142-a611-4a18-8673-c261edd1631c",
   "metadata": {
    "tags": []
   },
   "source": [
    "ReAct is an agent based chat mode built on top of a query engine over your data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d9dcfd5-ec96-4407-a5b7-ca3fadf22419",
   "metadata": {
    "tags": []
   },
   "source": [
    "For each chat interaction, the agent enter a ReAct loop:\n",
    "* first decide whether to use the query engine tool and come up with appropriate input\n",
    "* (optional) use the query engine tool and observe its output\n",
    "* decide whether to repeat or give final response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d765ea5c-e193-4298-96cc-978c87ffcafe",
   "metadata": {},
   "source": [
    "This approach is flexible, since it can flexibility choose between querying the knowledge base or not.\n",
    "However, the performance is also more dependent on the quality of the LLM. \n",
    "You might need to do more coercing to make sure it chooses to query the knowledge base at right times, instead of hallucinating an answer."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70be3c76-98d4-4df5-9cc6-8ceba34f23fb",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Get started in 5 lines of code"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8841bc6e-a196-4416-b886-31ea93e33d13",
   "metadata": {
    "tags": []
   },
   "source": [
    "Load data and build index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8bf4cb33-46c7-4f45-a43c-b478f68e9b0b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/suo/miniconda3/envs/llama/lib/python3.9/site-packages/deeplake/util/check_latest_version.py:32: UserWarning: A newer version of deeplake (3.6.7) is available. It's recommended that you update to the latest version using `pip install -U deeplake`.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext\n",
    "from langchain.llms import OpenAI\n",
    "\n",
    "service_context = ServiceContext.from_defaults(llm=OpenAI())\n",
    "data = SimpleDirectoryReader(input_dir=\"../data/paul_graham/\").load_data()\n",
    "index = VectorStoreIndex.from_documents(data, service_context=service_context)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0df6a939-9f2c-44cc-ad9d-8203029e8fbd",
   "metadata": {
    "tags": []
   },
   "source": [
    "Configure chat engine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "05eb7917-998c-4c36-b41a-84d29d3468c9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "chat_engine = index.as_chat_engine(chat_mode=\"react\", verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bbfb7bb-827d-4968-b3ae-7c9f92224e8c",
   "metadata": {},
   "source": [
    "Chat with your data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0a618731-2f1f-4379-8dac-6dc3564961c1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new  chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m\n",
      "Thought: Do I need to use a tool? Yes\n",
      "Action: Query Engine Tool\n",
      "Action Input: What did Paul Graham do in the summer of 1995?\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m\n",
      "In the summer of 1995, Paul Graham worked on building an application service provider (ASP) to allow people to edit code on a server through a browser, and then host the resulting applications for them. He recruited Dan Giffin, who had worked for Viaweb, and two undergrads who wanted summer jobs, and they got to work trying to build the software.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m Do I need to use a tool? No\n",
      "AI: That is correct, Paul Graham worked on building an application service provider (ASP) to allow people to edit code on a server through a browser, and then host the resulting applications for them.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "response = chat_engine.chat(\n",
    "    \"Use the tool to answer: what did Paul Graham do in the summer of 1995?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a7c5c258-397d-47f2-87d3-7c1e62138211",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "That is correct, Paul Graham worked on building an application service provider (ASP) to allow people to edit code on a server through a browser, and then host the resulting applications for them.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "03df3bfd-bc59-4f21-8458-f18b16e7b389",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new  chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m\n",
      "Thought: Do I need to use a tool? No\n",
      "AI: You asked me what Paul Graham did in the summer of 1995.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "response = chat_engine.chat(\"What did I ask you?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "58d4f135-8108-44ea-89fe-8d190089e089",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You asked me what Paul Graham did in the summer of 1995.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac91c351-d6ec-4958-9290-ad3078e7abe3",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Customize LLM"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60b87162-1a18-4390-b9a5-ddec6c1d1e36",
   "metadata": {},
   "source": [
    "Use ChatGPT (\"gpt-3.5-turbo\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "29bb0f40-3173-474f-8f78-a5b82f4e5645",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index import ServiceContext\n",
    "from langchain.chat_models import ChatOpenAI\n",
    "\n",
    "service_context = ServiceContext.from_defaults(\n",
    "    llm=ChatOpenAI(temperature=0.0, model=\"gpt-3.5-turbo\")\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aae7f923-cbd5-40af-ba2e-d3c2e3babcbf",
   "metadata": {},
   "source": [
    "Configure chat engine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "fa0a82b1-76db-4ba1-a6c3-b71d087ba132",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "chat_engine = index.as_chat_engine(\n",
    "    service_context=service_context, chat_mode=\"react\", verbose=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "cf5d2e49-04f8-4a61-9a30-ebc227d87d41",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new  chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What did Paul Graham do in the summer of 1995?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mBased on the given context information, it is not mentioned what Paul Graham did in the summer of 1995.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Final Answer\",\n",
      "    \"action_input\": \"Based on the available information, it is not mentioned what Paul Graham did in the summer of 1995.\"\n",
      "}\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "response = chat_engine.chat(\n",
    "    \"Use the tool to answer: what did Paul Graham do in the summer of 1995?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "8c6a387e-afdc-4829-852f-6916499a7352",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Based on the available information, it is not mentioned what Paul Graham did in the summer of 1995.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "48d8f1c6-ad3e-49fa-b365-a7404cc25f96",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new  chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What did Paul Graham do in the summer of 1995?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mBased on the given context information, it is not mentioned what Paul Graham did in the summer of 1995.\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 1.0 seconds as it raised ServiceUnavailableError: The server is overloaded or not ready yet..\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Final Answer\",\n",
      "    \"action_input\": \"Based on the available information, it is not mentioned what Paul Graham did in the summer of 1995.\"\n",
      "}\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "response = chat_engine.chat(\"What did I ask you before?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "1d5c0588-ed90-4930-92d8-89d8830b120c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Based on the available information, it is not mentioned what Paul Graham did in the summer of 1995.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42c4a975-8034-447f-a5fe-1e71e59f02c0",
   "metadata": {},
   "source": [
    "Reset chat engine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "c1326c7a-8056-47b9-924d-4a836a501312",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "chat_engine.reset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "365074a3-b9b3-4b73-a5a1-230267379175",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new  chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What did I ask you before?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mBased on the given context information, it is not possible to determine what question you asked before.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m{\n",
      "    \"action\": \"Query Engine Tool\",\n",
      "    \"action_input\": \"What is the response to my last comment?\"\n",
      "}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mThe given context information does not provide any information about a previous comment or response. Therefore, it is not possible to determine the response to the last comment.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "response = chat_engine.chat(\"What did I ask you before?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "37a387f8-4b90-4a73-9ffc-3960e6dbac5b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Agent stopped due to iteration limit or time limit.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c91b355b-ff05-4dfa-9fd8-e548a48e71b1",
   "metadata": {},
   "source": [
    "### Customize Personality (via System Prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ca5705e-f374-4734-9eeb-10a44b01c3c4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.prompts.system import SHAKESPEARE_WRITING_ASSISTANT\n",
    "\n",
    "chat_engine = index.as_chat_engine(\n",
    "    chat_mode=\"react\",\n",
    "    system_prompt=SHAKESPEARE_WRITING_ASSISTANT,\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "4621e1ab-3f45-420a-a687-12691da0c7a3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new  chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3m\n",
      "Thought: Do I need to use a tool? No\n",
      "AI: 'Tis a tale of woe, of a star-crossed lovers Romeo and Juliet. They be of two feuding families, and yet their hearts were true. Fate had other plans, and in the end, only sadness and despair remained.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "response = chat_engine.chat(\"Hi, tell me a tragic story\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
